Image signal processing device, digital camera and computer program product for processing image signal

ABSTRACT

R, G and B signals are output from individual pixels provided at an image-capturing element. Pixel output averages corresponding to the R, G and B signals are calculated for each of partition areas into which the light-receiving surface of the image-capturing element is divided. Ratios of the R and B pixel output averages to the G pixel output average are calculated for each partition area. Then, any area with the ratios of the pixel output averages within a predetermined range is extracted from the plurality of partition areas. R and B white balance gains are calculated based upon the total sums of the pixel output averages corresponding to the individual colors in the extracted area. A white balance adjustment is then executed by multiplying the values of R and B pixel outputs by the corresponding white balance gains.

INCORPORATION OF REFERENCE

[0001] The disclosures of the following priority applications are hereinincorporated by reference:

[0002] Japanese Patent Application No. 12-374840 filed Dec. 8, 2000

[0003] Japanese Patent Application No. 12-374841 filed Dec. 8, 2000

BACKGROUND OF THE INVENTION

[0004] 1. Field of the Invention

[0005] The present invention relates to a device that processes an imagesignal input from a color image-capturing element, a digital camera anda computer program product utilized in image signal processing, which,more specifically, achieve an improvement in the white balanceperformance.

[0006] 2. Description of the Related Art

[0007] There are image signal processing devices such as electronicstill cameras and video cameras in the known art that implement whitebalance adjustment in order to achieve faithful reproduction ofwhite-color areas of the subject by processing image signals obtainedthrough an image-capturing operation executed at a color image-capturingelement (see Japanese Patent Publication No. 2997234, for instance).

[0008]FIG. 14 presents a control block diagram of an image signalprocessing circuit achieved in the related art. Now, a brief explanationis given on the white balance adjustment implemented in the related artin reference to FIG. 14. This camera includes an image-capturing deviceadopting the TTL system and an image signal processing device. A subjectimage formed on a color image-capturing element 52 by a photographiclens 51 is converted to an electrical image signal by the colorimage-capturing element 52. A brightness signal Y is generated from theimage signal at a brightness signal processing unit 53 and alow-frequency component YL of the brightness signal, a red-color signalR and a blue-color signal B are generated from the image signal at achroma signal processing unit 54. The brightness signal Y is a signalachieved by mixing red R, green G and blue B at a ratio satisfying theexpression below.

Y=0.30 R+0.59 G+0.11 B  (1)

[0009] The red-color signal R is multiplied by a white balance gainRgain for the red-color signal and, as a result, a red-color signal R′is generated at an R gain control unit 55. The blue-color signal B ismultiplied by a white balance gain Bgain for the blue-color signal and,as a result, a blue-color signal B′ is generated at a B gain controlunit 56. Next, a color difference signal R−Y is generated at a matrixamplifier 57 based upon the red-color signal R′ and the low-frequencycomponent YL of the brightness signal. In addition, a color differencesignal B−Y is generated at a matrix amplifier 58 based upon theblue-color signal B′ and the low-frequency component YL of thebrightness signal. The color difference signals are expressed as

R−Y=0.70 R−0.59 G−0.11 B  (2)

B−Y=0.89 B−0.59 G−0.30 R  (3)

[0010] These color different signals R−Y and B−Y are provided to acompression processing unit 59 where they are compressed in the JPEGformat before they are recorded into a recording medium.

[0011] Averaging units 60 and 61 respectively ascertain the averages ofthe color difference signals R−Y and B−Y for the entire image plane. Acontrol voltage calculation unit 62 calculates white balance gains Rgainand Bgain that will set the levels of the average signals to 0, i.e.,the white balance gains Rgain and Bgain that satisfy R=B=G. As describedabove, the individual red-color signals R and blue-color signals B arerespectively multiplied by the white balance gains Rgain and Bgain for awhite balance adjustment.

[0012] However, there is a problem with the image signal processingdevice in the related art in that an accurate white balance adjustmentcannot be achieved when photographing a scene with a subject having achromatic color occupying a wide area of the photographic image plane,since the white balance adjustment operation is subjected to thepredominant influence of the color of the subject on the photographicscene.

[0013] An object of the present invention is to achieve a good whitebalance even when a subject having a chromatic color takes up a widearea of the photographic image plane.

SUMMARY OF THE INVENTION

[0014] The image signal processing device that processes an image signalconstituting a subject image captured at an image-capturing elementcomprises a saturation calculating unit that calculates a saturationlevel in each of a plurality of partition areas based upon pixel outputsfrom the partition areas into which the light-receiving surface of theimage-capturing element is divided, an area extracting unit thatextracts an area having a saturation level within a predetermined rangeand a white balance adjustment unit that performs a white balanceadjustment based upon pixel outputs corresponding to the individualcolors from the extracted area.

[0015] The saturation calculating unit may include an averagecalculating unit that calculates the pixel output averages for theindividual colors based upon the pixel signals output in correspondenceto the individual partition areas and an average ratio calculating unitthat calculates a ratio of the pixel output average corresponding toanother color to the pixel output average of a reference color for eachpartition area. In this case, the area extracting unit should extract anarea with the ratio of the pixel output averages within a predeterminedrange from the plurality of partition areas.

[0016] The present invention may be constituted such that an area to beused for a white balance adjustment is selected based upon the hue ofthe subject image, instead.

[0017] In such a case, the image signal processing device according tothe present invention that processes an image signal constituting asubject image captured at an image-capturing element comprises a huecalculating unit that calculates the hue of each of a plurality ofpartition areas based upon pixel outputs from partition areas into whichthe light-receiving surface of the image-capturing element is divided,an area extracting unit that extracts partition area with a huemanifesting a frequency equal to or lower than a predetermined frequencyvalue from the plurality of partition areas based upon a hue frequencydistribution among the plurality of partition areas and a white balanceadjustment unit that performs a white balance adjustment based uponpixel outputs corresponding to the various colors from the extractedarea.

[0018] The hue calculating unit may include an average calculating unitthat calculates the pixel output averages for the individual colorsbased upon the pixel signals output in correspondence to the individualpartition areas, an average ratio calculating unit that calculates aratio of the pixel output average corresponding to another color to thepixel output average of a reference color for each partition area and ahue detecting unit that detects the hue of each partition area basedupon the ratio of the pixel output averages.

[0019] The white balance adjustment unit that executes the white balanceadjustment based upon the saturation levels or hues may include a totalsum calculating unit that calculates the total sums of the pixel outputaverages corresponding to the individual colors in the extracted area, again calculating unit that calculates a white balance gain for a colorother than the reference color based upon the total sums correspondingto the individual colors and a pixel output adjustment unit thatimplements a white balance adjustment by multiplying the pixel outputscorresponding to a color other than the reference color by the whitebalance gain.

[0020] The present invention may be adopted in a digital camera havingthe image signal processing device described above and a recordingimage-capturing element that outputs an image signal for recording thecaptured subject image into a recording medium. In such a digitalcamera, the white balance adjustment may be executed by using the imagesignal output by the recording image-capturing element. If the digitalcamera is also provided with a photometering image-capturing elementthat outputs photometering signals indicating the subject brightnesslevels in the individual photometering areas into which the photographicfield is divided, the white balance adjustment may be implemented byusing an image signal from the photometering image-capturing element.

[0021] The present invention may be adopted in a computer-readablecomputer program product that includes a program for processing an imagesignal constituting a subject image captured at an image-capturingelement. The program in this case comprises saturation calculationprocessing in which the level of saturation is calculated for each of aplurality of partition area based upon pixel outputs from the partitionareas into which the light-receiving surface of the image-capturingelement is divided, area extraction processing in which an area having asaturation level within a predetermined range is extracted from theplurality of partition areas and white balance adjustment processing inwhich white balance adjustment is performed based upon pixel outputscorresponding to the various colors from the extracted area.

[0022] The saturation calculation processing may include averagecalculation processing in which pixel output averages are calculated forthe individual colors based upon pixel signals output in correspondenceto the individual partition areas and average ratio calculationprocessing in which a ratio of the pixel output average corresponding toanother color to the pixel output average corresponding to a referencecolor is calculated for each partition area. In such a case, any areawhere the ratio of the pixel output averages is within a predeterminedrange is extracted from the plurality of partition areas.

[0023] If an area to be used for white balance adjustment is selectedbased upon the hue of the subject image, the image signal processingprogram in the computer-readable computer program product according tothe present invention comprises hue calculation processing in which thehue of each of a plurality of partition areas is calculated based uponpixel outputs from the partition areas into which the light-receivingsurface of the image-capturing element is divided, area extractionprocessing in which any partition area with a hues manifesting afrequency equal to or lower than a predetermined frequency value isextracted from the plurality of partition areas based upon a huefrequency distribution among the plurality of partition areas and whitebalance adjustment processing in which a white balance adjustment isexecuted based upon pixel outputs corresponding to various colors fromthe extracted area.

[0024] The hue calculation processing may include average calculationprocessing in which pixel output averages are calculated for theindividual colors based upon pixel signals output in correspondence tothe individual partition areas into which the light-receiving surface ofthe image-capturing element is divided, average ratio calculationprocessing in which a ratio of the pixel output average corresponding toanother color to the pixel output average corresponding to a referencecolor is calculated for each partition area and hue detection processingin which the hue of each partition area is detected based upon the ratioof the pixel output average values.

[0025] The white balance processing executed based upon saturationlevels or hues may include total sum calculation processing in which thetotal sums of the pixel output averages corresponding to the individualcolors in the extracted area are calculated, gain calculation processingin which a white balance gain for a color other than the reference coloris calculated based upon the total sums corresponding to the individualcolors and pixel output adjustment processing in which a white balanceadjustment is executed by multiplying the pixel outputs corresponding toa color other than the reference color by the white balance gain.

[0026] The computer program product may be realized as a recordingmedium having recorded therein the program or a carrier wave on whichthe program is embodied as a data signal.

[0027] In the computer program product described above, too, the imagesignal utilized for white balance adjustment may be an image signal forrecording the captured subject image into a recording medium or it maybe photometering signals indicating the subject brightness levels in theindividual photometering areas into which the photographic field isdivided.

[0028] The pixel outputs from the image-capturing element may include ared-color component, a green-color component and a blue color componentor may include a green-color component, a yellow-color component, acyan-color component and a magenta-color component. In either case, thegreen color is the reference color.

BRIEF DESCRIPTION OF THE DRAWINGS

[0029]FIG. 1 is a cross sectional view of an embodiment achieved bymounting the image signal processing device according to the presentinvention at a camera;

[0030]FIG. 2 presents a block diagram of the control circuits of thecamera shown in FIG. 1;

[0031]FIG. 3 shows an example of the R, G and B pixel arrangement thatmay be adopted at color image-capturing element;

[0032]FIG. 4 shows a pixel array with complementary color filters at thecolor image-capturing element;

[0033]FIG. 5 presents a flowchart of an example of the image processing;

[0034]FIG. 6 presents a flowchart of the image processing incontinuation from FIG. 5;

[0035]FIG. 7 shows the pixel average ratios RG and BG in the cameraachieved in a first embodiment, mapped on a plane;

[0036]FIG. 8 shows the pixel average ratios RG and BG in the cameraachieved in a second embodiment, mapped on a plane;

[0037]FIG. 9 presents a histogram in correspondence to various rotatingangle ranges in for the camera achieved in the second embodiment;

[0038]FIG. 10 shows step S9A executed instead of step S9 in FIG. 5;

[0039]FIG. 11 presents a flowchart of the white balance gain calculationprocessing implemented in a third embodiment, which corresponds to theflowchart presented in FIGS. 5 and 6;

[0040]FIG. 12 presents a flowchart of an example of the image processingprogram that may be utilized when executing the white balance adjustmentaccording to the present invention on a personal computer;

[0041]FIG. 13 illustrates various modes in which the image processingprogram may be provided to a personal computer; and

[0042]FIG. 14 shows the structure adopted in an image processing devicein the related art.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] -First Embodiment-

[0044] Now, in reference to FIGS. 1˜7, an embodiment achieved byadopting the present invention in a TTL single lens reflex electronicstill camera is explained. It is to be noted that the present inventionmay be adopted in any of various types of electronic still camerasemploying an electronic viewfinder in which an image is captured by acolor image-capturing element such as a CCD, video cameras, scanners andthe like, as well as in the TTL single lens reflex electronic stillcamera.

[0045] In FIG. 1, the TTL single lens reflex camera includes a camerabody 1 and an interchangeable lens barrel 2 mounted at the camera body1. Light from a subject enters the lens barrel 2 and is guided to thecamera body 1 after passing through a photographic lens 3 and an openingof an aperture 4 inside the lens barrel 2.

[0046] A main mirror 5 is provided at the camera body 1. The main mirror5 is set at the position indicated by the dotted line unless the camerais engaged in a photographing operation, and the subject light isreflected at the main mirror 5 and is guided to a finder mat 6 where asubject image is formed. The subject image is then guided to an eyepiecelens 8 by a pentaprism 7 and is observed by the photographer via theeyepiece lens 8.

[0047] The subject image formed on the finder mat 6 is guided by thepentaprism 7 and a prism 9 to an image-forming lens 10 which re-formsthe subject image on a photometering image-capturing element 11. Thephotometering image-capturing element 11, which is constituted of aphotodiode or a CCD, measures the subject brightness in each of aplurality of photometering areas into which the photographic field isdivided.

[0048] The main mirror 5 is moved to the position indicated by the solidline for a photographing operation. Subject light having beentransmitted through a sub-mirror 12 passes through a released shutter 13and is guided to a photographing image-capturing element 14 where asubject image is formed. Some of the subject light is reflected by thesub-mirror 12 and is guided to a focal point detection device (notshown) installed at the bottom of the camera body 1 to enable detectionof the focal adjustment state of the photographic lens 3.

[0049] At the photographing image-capturing element (CCD) 14, aplurality of photoelectric conversion elements, i.e., pixels, arearrayed on a plane. A red R, green G or blue B color filter is providedon each pixel, as shown in FIG. 3. An image-capturing element having acomplementary color filters for green G, yellow Ye, cyan Cy and magentaMa as illustrated in FIG. 4 may be employed instead. An image-capturingelement having such color filters is referred to as a colorimage-capturing element in this specification.

[0050] A CPU 21 in FIG. 2 controls the overall operations in the camera.The CPU 21 drives the photographing color image-capturing element (CCD)14 described above by controlling a timing generator (TG) 22 and adriver 23. In addition, the CPU 21 controls the timing generator 22 tocontrol an analog signal processing circuit 24, an A/D converter 25, animage processing circuit (ASIC) 26 and a buffer memory 27.

[0051] The analog signal processing circuit 24 implements analogprocessing such as gain control and noise reduction on analog R, G and Bsignals input from the color image-capturing element 14. The A/Dconverter 25 converts the analog R, G and B signals to digital signals.The image processing circuit (ASIC) 26 implements processing such aswhite balance adjustment, γ correction and interpolation contourcorrection on the digital R, G and B signals. The operation executed bythe image processing circuit 26 is to be detailed later.

[0052] Image data undergoing the image processing or having undergonethe processing are stored into the buffer memory 27. A compressioncircuit (JPEG) 28 compresses the image data stored in the buffer memory27 at a predetermined rate through the JPEG method. A display imagegenerating circuit 29 generates display image data from the image datastored in the buffer memory 27 and displays the display image at amonitor 30. The compressed image data are recorded into a memory card 31which may be constituted of, for instance, a flash memory. A half pressswitch 32 and a full press (shutter release) switch 33 provided inconjunction with a shutter release button are connected to the CPU 21.

[0053] Each time the shutter 13 is released and a photograph is taken,the CPU 21 controls the color image-capturing element 14 to storeelectrical charges and the read out the stored charges. The read outstored charges then undergo the signal processing at the analog signalprocessing circuit 24 and the digital conversion at the A/D converter25. Therefore, they are processed at the image processing circuit 26.Then, the captured image is displayed at the monitor 30, the image iscompressed at the compression circuit 28 and the compressed image isrecorded into the memory card 31.

[0054] The image processing circuit (ASIC) 26, constituted of a CPU andits peripheral components, processes R, G and B image signals byexecuting the image processing program shown in FIGS. 5 and 6. It is tobe noted that while the image processing is executed in software at amicrocomputer in the embodiment, the image processing may be executed inhardware instead.

[0055] In step S1, individuals sets of pixel data corresponding to R, Gand B obtained by converting the outputs from the color image-capturingelement 14 to digital signals at the A/D converter 25 are input. In thefollowing step S2, a weighted average of a plurality of sets of pixeldata to be used as an optical black is subtracted from the individualsets of pixel data in each line in the pixel array. In step S3, theindividual sets of pixel data corresponding to R, G and B in each lineof the pixel array are uniformly multiplied by a predetermined gain fora signal level adjustment and also for a correction of any inconsistencyof the sensitivity levels of the R and B pixels relative to thesensitivity of the G pixels.

[0056] In step S4, the RGB pixel data having undergone the processingdescribed above are output to the buffer memory 27 for storage, and insteps S5˜S11, white balance gains Rgain and Bgain are calculated basedupon the RGB pixel data having undergone the processing.

[0057] Now, the method adopted to calculate the white balance gainsRgain and Bgain is explained. First, the light-receiving surface of theimage-capturing element 14 having a plurality of pixels arrayed on planeis divided into a plurality of areas each containing at least two pixelsin step S5. This area division may be achieved through one of thefollowing methods.

[0058] 1. A method achieved by dividing the light-receiving surface intoareas containing equal quantities of pixels to one another in principle,while disallowing any overlapping of the divided areas.

[0059] 2. A method achieved by dividing the light-receiving surface intoareas containing equal quantities of pixels to one another in principle,while allowing overlapping of the divided areas.

[0060] 3. A method achieved by dividing the light-receiving surface intoareas containing different quantities of pixels, with areas at thecenter of the photographic image plane having a small quantity of pixelsand areas in the periphery of the photographic image plane having alarge quantity of pixels, i.e., with small central areas and largerperipheral areas.

[0061] 4. A method achieved by dividing the light-receiving surface intolarge areas for an initial arithmetic operation and then dividing thelight-receiving surface into smaller areas based upon the results of thearithmetic operation, i.e., a method in which the area size is changedbased upon the results of the arithmetic operation.

[0062] 5. A method achieved by combining the methods above.

[0063] The method that may be employed for area division is not limitedto that adopted in the embodiment.

[0064] The division method 2, which allows overlapping of the areas,enables calculation of more accurate white balance gains compared to thedivision method 1, which does not allow overlapping. In the divisionmethod 3, small areas are set at the center and larger areas are set atthe periphery since a main subject to be photographed is present at thecentral area of the photographic image plane with greater frequency. Itis possible to achieve more accurate white balance gains by executingmore precise arithmetic processing on the central areas containing themain subject than on the peripheral areas containing the background suchas the sky. By adopting the division method 4, in which the arithmeticoperation is initially performed by dividing the light-receiving surfaceinto large areas and an arithmetic operation is re-executed by dividingthe light-receiving surface into smaller areas if reliable resultscannot be obtained through the initial arithmetic operation, the lengthof time required for the arithmetic processing can be reduced.

[0065] Individual areas obtained by dividing the light-receiving surfaceinto n areas through any of the methods described above are assignedwith area numbers i (=1˜n) for identification. In step S6, averagesRa(i), Ga(i) and Ba(i) of pixel values corresponding to R, G and B ineach area i are calculated. A pixel value represents the level of thevoltage output from a given pixel, i.e., a brightness value. The voltagelevels may be within a range of 0˜255, for instance. In step S7, a ratioRG(i) of the R pixel average Ra(i) to the G pixel average Ga(i) and aratio BG(i) of the B pixel average Ba(i) to the G pixel average Ga(i) ineach area i are calculated through the following formulae.

RG(i)={Ra(i)−Ga(i)}/Ga(i)  (4)

BG(i)={Ba(i)−Ga(i)}/Ga(i)  (5)

[0066] In step S8, a point {RG(i), BG(i)} which is determined by theratio RG(i) and the ratio BG(i) in each area i obtained through thearithmetic operation is plotted on a plane with its horizontal axisrepresenting the ratio BG of the B pixel average to the G pixel averageand its vertical axis representing the ratio RG of the R pixel averageto the G pixel average, as shown in FIG. 7.

[0067] The position at which the point {RG(i), BG(i)} corresponding tothe area i is plotted on the plane coordinate system of the ratio RG ofthe R pixel average to the G pixel average and the ratio BG of the Bpixel average to the G pixel average represents the level of saturationof the subject in the area i. For instance, if blue sky is presentwithin the photographic image plane, the ratio BG in an area containingthe blue sky is high, whereas the ratio RG becomes higher whenphotographing a red rose garden. It is difficult to achieve good whitebalance when photographing a scene with a chromatic subject with a highsaturation level taking up a large area of the photographic image planedue to the intense effect of the color of the subject. Accordingly, itis judged that a chromatic subject with a high saturation level ispresent in an area with the ratios RG and BG of the pixel averagesexceeding a predetermined value K and this area is excluded from thewhite balance gain calculation in the embodiment. Consequently, thewhite balance adjustment is implemented by excluding a chromatic subjectwith a high saturation level from the photographic image plane, whichmakes it possible to obtain good adjustment results.

[0068] More specifically, in step S9, any areas i with the correspondingpoints {RG(i), BG(i)} present inside a circle with a radius K around thecenter (0, 0) on the plane shown in FIG. 7 are extracted. Namely, areasi satisfying the following expression are extracted.

{square root}{square root over ( )}{|RG(i)|² +|BG(i)|² }≦K  (6)

[0069] In step S10, the total sums of the averages corresponding to R, Gand B pixels in all extracted areas are calculated through the followingformulae.

Rt=ρRa(i)  (7)

Gt=ΣGa(i)  (8)

Bt=ΣBa(i)  (9)

[0070] In step S11, the white balance gains Rgain and Bgain arecalculated through the following formulae based upon the total sums Rt,Bt and Gt.

Rgain=Rt/Gt  (10)

Bgain=Bt/Gt  (11)

[0071] Once the white balance gains Rgain and Bgain are calculated, theRGB pixel data stored earlier into the buffer memory 27 are input fromthe buffer memory 27 in step S12. Then, a white balance adjustment isexecuted by multiplying all the R pixel values by the gain Rgain and bymultiplying all the B pixel values by the gain Bgain.

[0072] In step S14 following the white balance adjustment, black leveladjustment and γ correction in the known art are executed. Next, in stepS15, interpolation/contour correction processing in the known art isperformed in units of areas each containing p (down)×q (across) pixels.Namely, the image data having undergone the white balance adjustment areformatted in preparation for JPEG data compression in units ofindividual sets of block data each corresponding to a pixel area of p×qpixels and aγ signal for an areas of p1×q1 pixels and a Cb signal and aCy signal corresponding to areas of p2×q2 pixels are generated as aresult. Since the black level adjustment, the γ correction, theinterpolation/contour correction processing described above are of theknown art and do not bear direct relevance to the present invention,their detailed explanation is omitted. Lastly, the Y signal, the Cbsignal and the Cy signal are output to the buffer memory 27 for storagein step S16.

[0073] As explained above, in the first embodiment;

[0074] (1) the light-receiving surface of the color image-capturingelement 14 is divided into a plurality of areas i,

[0075] (2) the pixel output averages Ra(i), Ga(i) and Ba(i)corresponding to R, G and B colors in each area i are calculated,

[0076] (3) the ratios RG(i) and BG(i) of the pixel output averages Ra(i)and Ba(i) corresponding to the other colors R and B to the pixel outputaverage Ga(i) corresponding to the green color G are calculated for eacharea i,

[0077] (4) any area having the ratios RG(i) and BG(i) of the pixeloutput averages indicating a point within the predetermined range K isextracted from the plurality of areas, and

[0078] (5) a white balance adjustment is executed based upon the pixeloutputs corresponding to the individual colors in the extracted area.

[0079] As a result, good white balance is achieved even when a chromaticsubject takes up a large area of the photographic image plane for thereason detailed below.

[0080] As explained earlier, the position of the point {RG(i), BG(i)} ofa given area i on the plane coordinates of the ratio RG of the R pixelaverage to the G pixel average and the ratio BG of the B pixel averageto the G pixel average indicates the saturation level of the subject inthe area i. For instance, an area containing blue sky in thephotographic image plane has a larger ratio BG, whereas the ratio RGbecomes higher when photographing a rose garden with red roses. Goodwhite balance cannot be achieved when photographing a scene with achromatic subject with a high saturation level taking up a large area ofthe photographic image plane due to the intense effect of the color inthe subject of the photographic scene.

[0081] In the embodiment, an area with either the ratio RG or the ratioBG of the pixel averages exceeding the predetermined value K is judgedto contain a chromatic subject with a high saturation level and such anarea is excluded from the white balance gain calculation. In otherwords, areas achieving saturation levels within a predetermined range,i.e., saturation levels within the circle over the radius K as expressedin expression (6) , are extracted from the plurality of areas i, and awhite balance adjustment is executed based upon the pixel outputscorresponding to the individual colors from the extracted areas. As aresult, the white balance adjustment is executed by excluding achromatic subject with a high saturation level from the photographicimage plane and thus, good adjustment results are achieved.

[0082] It is to be noted that the predetermined value K is a referencevalue that may be determined as explained below. Namely, thepredetermined value K is set in advance so as to achieve good whitebalance adjustment for a photograph with a large chromatic areacontained in the photographic image plane by taking a large number oftest photographs each with a large chromatic area present within thephotographic image plane under various photographing conditions.

[0083] -Second Embodiment-

[0084] In the first embodiment, any area with the ratios RG and BG ofthe pixel averages exceeding the predetermined value K, as illustratedin FIG. 7, is judged to contain a chromatic subject with a highsaturation level and such an area is excluded from the white balancegain calculation. In the second embodiment, a histogram such as thatshown in FIG. 9 is prepared by ascertaining the frequency distributionof points {RG(i), BG(i)} corresponding to different rotating angleranges relative to the origin point (0, 0) on the plane coordinatesystem shown in FIG. 8, which is similar to that presented in FIG. 7,and any area to be excluded from the white balance gain calculation isdetermined by judging the presence/absence of a chromatic color in thearea based upon the histogram, instead.

[0085] In reference to the flowchart in FIGS. 5 and 6 and in theflowchart in FIG. 10, an explanation is now given by focusing on thedifference from the first embodiment. In step S8, a point {RG(i), BG(i)}determined by the ratio RG(i) and the ratio BG(i) of each area iobtained through the arithmetic operation is plotted on the planecoordinate system with its horizontal axis representing the ratio BG ofthe B pixel average to the G pixel average and its vertical axisrepresenting the ratio RG of the R pixel average to the G pixel average.In step S9A (see FIG. 10), a histogram such as that shown in FIG. 9 isprepared by ascertaining the frequency distribution of points {RG (i),BG (i)} in each rotating angle range relative to the origin point (0, 0)on the plane coordinate system in FIG. 8. It is to be noted that whilean explanation is given in reference to the embodiment on an example inwhich each rotating angle range represents a 10° angle, the anglerepresented by each rotating angle range may be other than 10°.

[0086] The position of the point {RG(i), BG(i)} corresponding to a givenarea i on the plane coordinate system of the ratio RG of the R pixelaverage to the G pixel average and the ratio BG of the B pixel averageto the G pixel average represents the identity of the chromatic color ofthe subject in the area i, i.e., the hue of the subject in the area i.The frequency distribution of points {RG(i), BG(i)} for each rotatingangle range indicates the subject hue tendency. For instance, the ratioBG becomes higher if blue sky takes up a large area in the photographicimage plane, resulting in a high frequency of points {RG(i), BG(i)}present in the rotating angle ranges around the horizontal axis BG. Theratio RG becomes higher when photographing a rose garden with many redroses in full bloom, and in such a case, the frequency of points {RG(i),BG(i)} present in the rotating angle ranges near the vertical axis RGincreases.

[0087] However, a good white balance cannot be achieved whenphotographing a scene with a chromatic subject taking up a large area ofthe photographic image plane as described above due to the intenseeffect of the color of the subject. Accordingly, an area i with a huemanufacturing with a value exceeding a predetermined frequency value KRis judged to contain a chromatic subject which takes up a large area ofthe photographic image plane and such an area is excluded from the whitebalance gain calculation in the embodiment. Consequently, the whitebalance adjustment can be executed by excluding the chromatic subjecttaking up a large area of the photographic image plane, which makes itpossible to achieve good adjustment results.

[0088] More specifically, any area i with a frequency value equal to orlower than the predetermined value KR in the histogram shown in FIG. 9is extracted in step S10, and then in the following step S11, the totalsums Rt, Gt and Bt of the averages of the R, G and B pixel values in allthe extracted areas are calculated through formulae (7)˜(9) presentedearlier. In step S12, white balance gains Rgain and Bgain are calculatedthrough formulae (10) and (11) based upon the total sums Rt, Gt and Bt.

[0089] Once the white balance gains Rgain and Bgain are calculated, theRGB pixel data stored earlier into the buffer memory 27 are input fromthe buffer memory 27 in step S12. Then, a white balance adjustment isexecuted by multiplying all the R pixel values by the gain Rgain and bymultiplying all the B pixel values by the gain Bgain.

[0090] As explained above, in the second embodiment;

[0091] (1) the light-receiving surface of the color image-capturingelement 14 is divided into a plurality of areas i,

[0092] (2) the pixel output averages Ra(i), Ga(i) and Ba(i)corresponding to R, G and B colors in each area i are calculated and theratios RG(i) and BG(i) of the pixel output averages Ra(i) and Ba(i)corresponding to the other colors R and B to the pixel output averageGa(i) corresponding to the reference color G are calculated for eacharea i,

[0093] (3) the hue of the area i is detected based upon the ratios RG(i)and BG(i) of the pixel output averages,

[0094] (4) any area with a hue manufacturing with a frequency equal toor lower than the predetermined frequency value KR is extracted from theplurality of areas i based upon the hue frequency distribution among theplurality of areas i and

[0095] (5) a white balance adjustment is executed based upon the pixeloutputs corresponding to the individual colors in the extracted area.

[0096] As a result, a good white balance is achieved even when achromatic subject takes up a large area of the photographic image planein the image data undergoing the white balance adjustment.

[0097] In the embodiment described above, the subject hue tendency ineach area is judged by setting a plane coordinate system of the ratio RGof the R pixel average to the G pixel average and the ratio BG of the Bpixel average to the G pixel average. However, the present invention maybe realized by using any coordinate system other than the coordinatesystem described above, such as an XYZ coordinate system, an LABcoordinate system, an LUV coordinate system or an HSC coordinate system,as long as the coordinate system represents hues.

[0098] In the first and second embodiments described above, the whitebalance adjustment is executed by using the outputs from thephotographing color image-capturing element 14. However, similaradvantages can be achieved by executing the white balance adjustmentexplained above with the outputs from the photometering colorimage-capturing element 11 as well.

[0099] An explanation has been given above in reference to theembodiment on an example in which the image-capturing element includesthree primary color filters corresponding to R, G and B colors withgreen color used as the reference color. However, the colors of thefilters, their arrangement and the color used for reference at the colorimage-capturing element are not limited to those in the embodiment, andsimilar advantages can be achieved by employing an image-capturingelement having, for instance, complementary color filters in G, Ye, Cyand Ma.

[0100] (Variation; when Employing an Image-capturing Element withComplementary Color Filters)

[0101] A white balance adjustment is executed as described below in acamera that employs an image-capturing element having complementarycolor filters, as illustrated in FIG. 4. In step S5 in FIG. 5, thelight-receiving surface is divided into a plurality of areas through thedivision method explained earlier. Next, in step S6, averages Ga(i),Ye₁₃ a(i), Cy_a(i) and Ma_a(i) of the pixel values respectivelycorresponding to green G, yellow Ye, cyan Cy and magenta Ma in eachpartition area i are calculated.

[0102] In step S7, a ratio YeG(i) of the Ye pixel average Ye_a(i) to theG pixel average Ga(i), a ratio CyG(i) of the Cy pixel average Cy_a(i) tothe G pixel average Ga(i) and a ratio MaG(i) of the Ma pixel averageMa_a(i) to the G pixel average Ga(i) in each area i are calculatedthrough the following formulae;

YeG(i)={Ye _(—) a(i)−2Ga(i)}/2Ga(i)  (12)

CyG(i)={Cy _(—) a(i)−2Ga(i)}/2Ga(i)  (13)

MaG(i)={Ma _(—) a(i)−2Ga(i)}/2Ga(i)  (14)

[0103] In step S8, a point: {YeG(i), CyG(i), MaG(i)} which is determinedby the ratios YeG(i), CyG(i) and MaG(i) corresponding to each area iobtained through the arithmetic operation is plotted on a spatialcoordinate system constituted of three axes with an X axis representingthe ratio YeG of the Ye pixel average to the G pixel average, a Y axisrepresenting the ratio CyG of the Cy pixel average to the G pixelaverage and a Z axis representing the ratio MaG of the Ma pixel averageto the G pixel average. In step S9, any area i with its point {YeG(i),CyG(i), MaG(i)} present within a sphere with a radius K′ around thecenter (0, 0, 0) on the space coordinate system is extracted. In otherwords, any area i that satisfies the following expression is extracted.

{square root}{square root over ( )}{|YeG(i)|² +|CyG(i)|² +|MaG(i)|²}≦K′  (15)

[0104] In step S10, the total sums of the averages of the pixel valuescorresponding to G, Ye, Cy and Ma in all the extracted areas arecalculated through the following formulae.

Gt=ΣGa(i)  (16)

Ye _(—) t=ΣYe _(—) a(i)  (17)

Cy _(—) t=ΣCy _(—) a(i)  (18)

Ma _(—) t=ΣMa _(—) a(i)  (20)

[0105] In step S11, white balance gains Ye_gain, Cy_gain and Ma_gain arecalculated through the following formulae based upon the total sums Gt,Ye_t, Cy_t and Ma_t.

Ye_gain=Ye _(—) t/2Gt  (21)

Cy_gain=Cy _(—) t/2Gt  (22)

Ma_gain=YMa _(—) t/2Gt  (23)

[0106] Once the white balance gains Ye_gain, Cy_gain and Ma_gain arecalculated, the complementary color pixel data having been stored inadvance at the buffer memory 27 are input from the buffer memory 27 anda white balance adjustment is executed by multiplying the valuescorresponding to all the Ye pixels by the gain Ye_gain, multiplying thevalues corresponding to all the Cy pixels by the gain Cy_gain andmultiplying the values corresponding to all the Ma pixels by the gainMa_gain in step S12.

[0107] An explanation is given above on the examples in which thepresent invention is adopted in an electronic still camera. However, rawimage data, which have been obtained through a photographing operationperformed in an electronic still camera, are taken into the personalcomputer and a white balance adjustment may be executed as describedabove on a personal computer. Such raw image data may be utilized by,for instance, recording image data having undergone the processing instep S3 in FIG. 5 into a recording medium.

[0108]FIG. 12 shows an example of an image processing program that maybe used to execute a white balance adjustment on a personal computer. Instep S101, raw image data recorded in a removable recording medium areinput to a storage area in, for instance, a hard disk of the personalcomputer. In step S102, white balance adjustment gains are calculated byexecuting the processing in steps S6˜S11 explained earlier in referenceto FIGS. 5 and 6, for instance. In step S103, the image data valuescorresponding to the individual pixels are multiplied by the gains thathave been calculated, as in step S13. Subsequently, the processing insteps S103˜S105 (corresponding to steps S13˜S15 explained earlier) isexecuted and then in step S106, an image having undergone the whitebalance adjustment is recorded into the removable recording medium.

[0109] When the present invention is adopted to execute a white balanceadjustment on a personal computer, the image processing program shown inFIG. 12 may be provided in a recording medium such as a CD-ROM orthrough a data signal on the Internet or the like.

[0110]FIG. 13 illustrates how this may be achieved. A computer 300receives the program via a CD-ROM 304. The personal computer 300 can beconnected with a communication line 301. A computer 302 is a servercomputer that provides the program stored in a recording medium such asa hard disk 303. The communication line 301 may be a communication linethrough which Internet communication or personal computer communicationmay be implemented or it may be a dedicated communication line. Thecomputer 302 reads out the program from the hard disk 303 and thentransmits the program to the personal computer 300 via the communicationline 301.

[0111] In other words, the program which is embodied as a data signal ona carrier wave is transmitted via the communication line 301. Thus, theprogram can be provided as a computer-readable computer program productin any of various modes such as a recording medium and a carrier wave.

[0112] The white balance adjustment processing in an electronic stillcamera is sometimes upgraded after the camera is marketed. In such acase, too, the image processing program shown in FIGS. 5 and 6 can bedistributed in a CD-ROM or through the internet.

What is claimed is;
 1. An image signal processing device that processesan image signal constituting a subject image captured by animage-capturing element, comprising: an average calculating unit thatcalculates pixel output averages for individual colors in each of aplurality of partition areas into which a light-receiving surface of theimage-capturing element is divided based upon the image signal; anaverage ratio calculating unit that calculates a ratio of a pixel outputaverage corresponding to another color to a pixel output average of areference color for each partition area; an area extracting unit thatextracts an area having a ratio of pixel output averages within apredetermined range; and a white balance adjustment unit that performs awhite balance adjustment based upon pixel outputs corresponding to theindividual colors from the extracted area.
 2. An image signal processingdevice according to claim 1, wherein: the white balance adjustment unitincludes; a total sum calculating unit that calculates total sums ofpixel output averages corresponding to the individual colors in theextracted area; a gain calculating unit that calculates a white balancegain for a color other than the reference color based upon the totalsums corresponding to the individual colors; and a pixel outputadjustment unit that performs white balance adjustment by multiplyingpixel outputs corresponding to the color other than the reference colorby the white balance gain.
 3. An image signal processing deviceaccording to claim 1, wherein: the pixel outputs include a red-colorcomponent, a green-color component and a blue color component and thereference color is green.
 4. An image signal processing device accordingto claim 2, wherein: the pixel outputs include a red-color component, agreen-color component and a blue color component and the reference coloris green.
 5. An image signal processing device according to claim 1,wherein: the pixel outputs include a green-color component, ayellow-color component, a cyan-color component and a magenta-colorcomponent and the reference color is green.
 6. An image signalprocessing device according to claim 2, wherein: the pixel outputsinclude a green-color component, a yellow-color component, a cyan-colorcomponent and a magenta-color component and the reference color isgreen.
 7. A digital camera having: an image signal processing deviceaccording to claim 1; and a recording image-capturing element thatoutputs an image signal used for recording the captured subject imageinto a recording medium, wherein: an image signal to be used for whitebalance adjustment is the image signal output by the recordingimage-capturing element.
 8. A digital camera having: an image signalprocessing device according to claim 1; a recording image-capturingelement that outputs an image signal for recording the captured subjectimage into a recording medium; and a photometering image-capturingelement that outputs a photometering signal indicating a subjectbrightness level in each of photometering areas into which aphotographic field is divided, wherein: an image signal to be used forwhite balance adjustment is the image signal output by the photometeringimage-capturing element.
 9. A computer-readable computer program producthaving a program to be used to process an image signal constituting asubject image captured at an image-capturing element, with the programcomprising: average calculation processing in which pixel outputaverages are calculated for individual colors in each of a plurality ofpartition areas into which a light-receiving surface of theimage-capturing element is divided based upon the image signal; averageratio calculation processing in which a ratio of the pixel outputaverage corresponding to another color to the pixel output averagecorresponding to a reference color is calculated for each partitionarea; area extraction processing in which any area having a ratio ofpixel output averages within a predetermined range is extracted from theplurality of partition areas; and white balance adjustment processing inwhich white balance adjustment is performed based upon pixel outputscorresponding to the individual colors from the extracted area.
 10. Acomputer program product according to claim 9, with the program furthercomprising: total sum calculation processing in which total sums of thepixel output averages corresponding to the individual colors in theextracted area are calculated; gain calculation processing in which awhite balance gain for a color other than the reference color iscalculated based upon the total sums corresponding to the individualcolors; and pixel output adjustment processing in which a white balanceadjustment is performed by multiplying pixel outputs corresponding tothe color other than the reference color by the white balance gain. 11.A computer program product according to claim 9, constituted as arecording medium having recorded therein the program.
 12. A computerprogram product according to claim 9, constituted as a carrier wavehaving the program embodied as a data signal thereupon.
 13. An imagesignal processing device according to claim 9, wherein: the pixeloutputs include a red-color component, a green-color component and ablue color component and the reference color is green.
 14. A computerprogram product according to claim 10, wherein: the pixel outputsinclude a red-color component, a green-color component and a blue colorcomponent and the reference color is green.
 15. A computer programproduct according to claim 9, wherein: the pixel outputs include agreen-color component, a yellow-color component, a cyan-color componentand a magenta-color component and the reference color is green.
 16. Acomputer program product according to claim 10, wherein: the pixeloutputs include a green-color component, a yellow-color component, acyan-color component and a magenta-color component and the referencecolor is green.
 17. A computer program product according to claim 9,wherein: an image signal to be utilized in white balance adjustment isan image signal used for recording the captured subject image into arecording medium.
 18. A computer program product according to claim 10,wherein: an image signal to be utilized in white balance adjustment is aphotometering signal indicating subject brightness levels eachcorresponding to one of a plurality of photometering areas into which aphotographic field is divided.
 19. An image signal processing devicethat processes an image signal constituting a subject image captured byan image-capturing element comprising: a saturation calculating unitthat calculates a saturation level in each of a plurality of partitionareas based upon pixel outputs from the partition areas into which alight-receiving surface of the image-capturing element is divided; anarea extracting unit that extracts any area having a saturation levelwithin a predetermined range from the plurality of partition areas; anda white balance adjustment unit that performs a white balance adjustmentbased upon pixel outputs corresponding to individual colors from theextracted area.
 20. A digital camera having: an image signal processingdevice according to claim 19; and a recording image-capturing elementthat outputs an image signal used for recording the captured subjectimage into a recording medium, wherein: an image signal to be used forwhite balance adjustment is the image signal output by the recordingimage-capturing element.
 21. A digital camera having: an image signalprocessing device according to claim 19; a recording image-capturingelement that outputs an image signal for recording the captured subjectimage into a recording medium; and a photometering image-capturingelement that outputs a photometering signal indicating a subjectbrightness level in each of photometering areas into which aphotographic field is divided, wherein: an image signal to be used forwhite balance adjustment is the image signal output by the photometeringimage-capturing element.
 22. A computer-readable computer programproduct having a program to be used to process an image signalconstituting a subject image captured at an image-capturing element,with the program comprising: saturation calculation processing in whicha saturation level in each of a plurality of partition areas iscalculated based upon pixel outputs from the partition areas into whicha light-receiving surface of the image-capturing element is divided;area extraction processing in which any area having a saturation levelwithin a predetermined range is extracted from the plurality ofpartition areas; and white balance adjustment processing in which whitebalance adjustment is performed based upon pixel outputs correspondingto individual colors from the extracted area.
 23. An image signalprocessing device that processes an image signal constituting a subjectimage captured at an image-capturing element, comprising: an averagecalculating unit that calculates pixel output averages for individualcolors in each of a plurality of partition areas into which alight-receiving surface of the image-capturing element is divided basedupon the image signal; an average ratio calculating unit that calculatesa ratio of the pixel output average corresponding to another color tothe pixel output average of a reference color for each partition area; ahue detecting unit that detects a hue of each partition area based uponthe ratio of the pixel output averages; an area extraction unit thatextracts any partition area with a hue manifesting a frequency equal toor lower than a predetermined frequency value from the plurality ofpartition areas based upon a hue frequency distribution among theplurality of partition areas; and a white balance adjustment unit thatperforms white balance adjustment based upon pixel outputs correspondingto individual colors from the extracted area.
 24. An image signalprocessing device according to claim 23, wherein: the white balanceadjustment unit further includes; a total sum calculating unit thatcalculates total sums of pixel output averages corresponding to theindividual colors in the extracted area; a gain calculating unit thatcalculates a white balance gain for a color other than the referencecolor based upon the total sums corresponding to the individual colors;and a pixel output adjustment unit that performs white balanceadjustment by multiplying pixel outputs corresponding to the color otherthan the reference color by the white balance gain.
 25. An image signalprocessing device according to claim 23, wherein: the pixel outputsinclude a red-color component, a green-color component and a blue colorcomponent and the reference color is green.
 26. An image signalprocessing device according to claim 24, wherein: the pixel outputsinclude a red-color component, a green-color component and a blue colorcomponent and the reference color is green.
 27. An image signalprocessing device according to claim 23, wherein: the pixel outputsinclude a green-color component, a yellow-color component, a cyan-colorcomponent and a magenta-color component and the reference color isgreen.
 28. An image signal processing device according to claim 24,wherein: the pixel outputs include a green-color component, ayellow-color component, a cyan-color component and a magenta-colorcomponent and the reference color is green.
 29. A digital camera having:an image signal processing device according to claim 23; and a recordingimage-capturing element that outputs an image signal used for recordingthe captured subject image into a recording medium, wherein: an imagesignal to be used for white balance adjustment is the image signaloutput by the recording image-capturing element.
 30. A digital camerahaving: an image signal processing device according to claim 23; arecording image-capturing element that outputs an image signal forrecording the captured subject image into a recording medium; and aphotometering image-capturing element that outputs a photometeringsignal indicating a subject brightness level in each of photometeringareas into which a photographic field is divided, wherein: an imagesignal to be used for white balance adjustment is the image signaloutput by the photometering image-capturing element.
 31. Acomputer-readable computer program product having a program to be usedto process an image signal constituting a subject image captured at animage-capturing element, with the program comprising: average valuecalculation processing in which pixel output averages for individualcolors in each of a plurality of partition areas into which alight-receiving surface of the image-capturing element is divided basedupon the image signal; average ratio calculation processing in which aratio of the pixel output average corresponding to another color to thepixel output average corresponding to a reference color is calculatedfor each partition area; hue detection processing in which a hue of eachpartition area is detected based upon the ratio of the pixel outputaverages; area extraction processing in which any partition area with ahue manifesting a frequency equal to or lower than a predeterminedfrequency value is extracted from the plurality of partition areas basedupon a hue frequency distribution among the plurality of partitionareas; and white balance adjustment processing in which a white balanceadjustment is performed based upon pixel outputs corresponding toindividual colors from the extracted area.
 32. A computer-readablecomputer program product according to claim 31 with the program furthercomprising: total sum calculation processing in which total sums of thepixel output averages corresponding to the individual colors in theextracted area are calculated; gain calculation processing in which awhite balance gain for a color other than the reference color iscalculated based upon the total sums corresponding to the individualcolors; and pixel output adjustment processing in which a white balanceadjustment is performed by multiplying pixel outputs corresponding tothe color other than the reference color by the white balance gain. 33.A computer program product according to claim 31, constituted as arecording medium having recorded therein the program.
 34. A computerprogram product according to claim 31, constituted as a carrier wavehaving the program embodied as a data signal thereupon.
 35. An imagesignal processing device according to claim 31, wherein: the pixeloutputs include a red-color component, a green-color component and ablue color component and the reference color is green.
 36. A computerprogram product according to claim 32, wherein: the pixel outputsinclude a red-color component, a green-color component and a blue colorcomponent and the reference color is green.
 37. A computer programproduct according to claim 31, wherein: the pixel outputs include agreen-color component, a yellow-color component, a cyan-color componentand a magenta-color component and the reference color is green.
 38. Acomputer program product according to claim 32, wherein: the pixeloutputs include a green-color component, a yellow-color component, acyan-color component and a magenta-color component and the referencecolor is green.
 39. A computer program product according to claim 31,wherein: an image signal to be utilized in white balance adjustment isan image signal used for recording the captured subject image into arecording medium.
 40. A computer program product according to claim 31,wherein: an image signal to be utilized in white balance adjustment is aphotometering signal indicating subject brightness levels eachcorresponding to one of a plurality of photometering areas into which aphotographic field is divided.
 41. An image signal processing devicethat processes an image signal constituting a subject image captured atan image-capturing, comprising: a hue calculating unit that calculates ahue of each of a plurality of partition areas based upon pixel outputsfrom the partition areas into which a light-receiving surface of theimage-capturing element is divided; an area extracting unit thatextracts any area having a hue manifesting a frequency equal to or lowerthan a predetermined frequency value from the plurality of partitionareas based upon a hue frequency distribution among the plurality ofpartition areas; and a white balance adjustment unit that performs awhite balance adjustment based upon pixel outputs corresponding to theindividual colors from the extracted area.
 42. A digital camera having:an image signal processing device according to claim 41; and a recordingimage-capturing element that outputs an image signal for recording thecaptured subject image into a recording medium, wherein: an image signalto be used for white balance adjustment is the image signal output bythe recording image-capturing element.
 43. A digital camera having: animage signal processing device according to claim 41; a recordingimage-capturing element that outputs an image signal for recording thecaptured subject image into a recording medium; and a photometeringimage-capturing element that outputs a photometering signal indicating asubject brightness level in each of photometering areas into which aphotographic field is divided, wherein: an image signal to be used forwhite balance adjustment is the image signal output by the photometeringimage-capturing element.
 44. A computer-readable computer programproduct having a program to be used to process an image signalconstituting a subject image captured at an image-capturing element,with the program comprising: hue calculation processing in which a hueof each of a plurality of partition areas is calculated based upon pixeloutputs from the partition areas into which a light-receiving surface ofthe image-capturing element is divided; area extraction processing inwhich any area manifesting a frequency equal to or lower than apredetermined frequency value is extracted from the plurality ofpartition areas based upon a hue frequency distribution among theplurality of partition areas; and white balance adjustment processing inwhich white balance adjustment is performed based upon pixel outputscorresponding to individual colors from the extracted area.